Method for testing the error ratio of a device

ABSTRACT

A method for testing the (Bit) Error Ration BER of a device against a maximal allowable (Bit) Error Ratio BER limit  with a early pass and/or early fail criterion, whereby the early pass and/or early fail criterion is allowed to be wrong only by a small probability D. ns bits of the output of the device are measured, thereby ne erroneous bits of the ns bits are detected. PD high  and/or PD low  are obtained, whereby PD high  is the worst possible likelihood distribution and PD low  is the best possible likelihood distribution containing the measured ne erroneous bits with the probablility D. The average numbers of erroneous bits NE high  and NE low  for PD high  and PD low  are obtained. NE high  and NE low  are compared with NE limit =BER limit ×ns. If NE limit  is higher than NE high  or NE limit  is lower than NE low  the test is stopped.

[0001] The application is related to testing the (Bit) Error Ratio of adevice, such as a digital receiver for mobile phones.

[0002] Concerning the state of the art, reference is made to U.S. Pat.No. 5,606,563. This document discloses a method of determining an errorlevel of a data channel comprised of receiving channel parity error dataindicating when bit errors occur within a set of data carried on thechannel (channel error events), successively integrating the channelerror events data over successive accumulation periods, comparing theintegrated channel error events data with a threshold, and indicating analarm in the event the integrated channel error events data exceeds thethreshold.

[0003] Testing digital receivers is two fold: The receiver is offered asignal, usually accompanied with certain, stress conditions. Thereceiver has to demodulate and to decode this signal. Due to stressconditions the result can be partly erroneous. The ratio of erroneouslydemodulated to correctly demodulated bits is measured in the Error Ratiotest. The Bit Error Ratio BER test or more general any error rate test,for example Frame Error Ratio FER, Block Error Ratio BLER eachcomprising a Bit Error test, is subject of this application.

[0004] BER testing is time consuming. This is illustrated by thefollowing example: a frequently used BER limit is 0.001. The bitratefrequently used for this test is approx. 10 kbit/s. Due to statisticalrelevance it is not enough to observe 1 error in 1000 bits. It is usualto observe approx. 200 errors in 200 000 bits. This single BER testlasts 20 seconds. There are combined tests which contain this single BERtest several times, e.g. the receiver blocking test. Within this testthe single BER test is repeated 12750 times with different stressconditions. Total test time here is more than 70 h. It is known in thestate of the art to fail the DUT (Device Under Test) early only, when afixed number of errors as 200 errors are observed before 200,000 bitsare applied. (Numbers from the example above).

[0005] It is the object of the present application to propose a methodfor testing the Bit Error Ratio of a device with reduced test time,preserving the statistical relevance.

[0006] The object is solved by the features of claim 1 or claim 4.

[0007] According to the present invention, the worst possible likelihooddistribution PD_(high) and/or the best possible likelihood distributionPD_(low) are obtained by the formula's given in claim 1 and 4,respectively. From these likelihood distributions, the average numberNE_(high) or NE_(low) is obtained and compared with the limitNE_(limit), respectively. If the limit NE_(limit) is higher than theaverage number NE_(high) or smaller than the average number NE_(low),the test is stopped and it is decided that the device has early passedor early failed, respectively.

[0008] The dependent claims contain further developments of theinvention.

[0009] The Poisson distribution preferably used for the presentinvention, is best adapted to the BER problem. The nature of the BitError occurrence is ideally described by the binomial distribution. Thisdistribution, however, can only be handled for small number of bits anderrors. For a high number of bits and a low Bit Error Ratio the binomialdistribution is approximated by the Poisson distribution. Thisprerequisite (high number of bits, low BER) is ideally fulfilled fornormal Bit Error Rate testing (BER limit 0.001) as long as the DUT isnot totally broken (BER 0.5). The application proposes to overcomeproblems with the discrete nature of the Poisson distribution. With thesame method an early pass condition as well as an early fail conditionis derived.

[0010] An embodiment of the invention is described hereafter. In thedrawings

[0011]FIG. 1 shows a diagram to illustrate the inventive method,

[0012]FIG. 2 shows the referenced Bit Error Ratio ber_(norm) as afunction of the measured errors ne,

[0013]FIG. 3 shows a-diagram to illustrate a measurement using a firstembodiment of the inventive method,

[0014]FIG. 4 shows a diagram to illustrate a measurement using a secondembodiment of the inventive method and

[0015]FIG. 5 shows a diagram illustrating the position at the end of thetest using the first embodiment of the inventive method as a function ofprobability.

[0016] In the following, the early fail condition and the early passcondition is derived with mathematical methods.

[0017] Due to the nature of the test, namely discrete error events, theearly stop conditions are declared not valid, when fractional errors<1are used to calculate the early stop limits. The application containsproposals, how to conduct the test at this undefined areas. Theproposals are conservative (not risky). A DUT on the limit does notachieve any early stop condition. The application proposes to stop thetest unconditioned at a specific number K (for example 200 bit) errors.As this proposal contains a pseudo paradox, an additional proposal toresolve this pseudo paradox is appended.

[0018] Based on a single measurement, a confidence range CR around thismeasurement is derived. It has the property that with high probabilitythe final result can be found in this range.

[0019] The confidence range CR is compared with the specified BER limit.From the result a diagram is derived containing the early fail and theearly pass condition.

[0020] With a finite number of samples ns of measured bits, the finalBit Error Ratio BER cannot be determined exactly. Applying a finitenumber of samples ns, a number of errors ne is measured. ne/ns ber isthe preliminary Bit Error Ratio.

[0021] In a single test a finite number of measured bits ns is applied,and a number of errors ne is measured. ne is connected with a certaindifferential probability in the Poisson distribution. The probabilityand the position in the distribution conducting just one single test isnot known.

[0022] Repeating this test infinite times, applying repeatedly the samens, the complete Poisson distribution is obtained. The average number(mean value) of errors is NE. NE/ns is the final BER. The Poissondistribution has the variable ne and is characterised by the parameterNE, the average or mean value. Real probabilities to find ne between twolimits are calculated by integrating between such limits. The width ofthe Poisson distribution increases proportional to SQR(NE), that means,it increases absolutely, but decreases relatively.

[0023] In a single test ns samples are applied and ne errors aremeasured. The result can be a member of different Poisson distributionseach characterized by another parameter NE. Two of them are given asfollows:

[0024] The worst possible distribution NE_(high), containing themeasured ne with the probability D₁, is given by $\begin{matrix}{D_{1} = {\int_{0}^{n\quad e}{{{PD}_{high}\left( {{NE}_{high},{ni}} \right)}{{ni}}}}} & (1)\end{matrix}$

[0025] In the example D₁ is 0.002=0.2% PD_(high) is, the wanted Poissondistribution with the variable ni. ne is the measured number of errors.

[0026] The best possible distributions NE_(low), containing the measuredne with the probability D₂ is given by $\begin{matrix}{D_{2} = {\int_{ne}^{\infty}{{{PD}_{low}\left( {{NE}_{low},{ni}} \right)}{{ni}}}}} & (2)\end{matrix}$

[0027] In the example D₂ is equal D₁ and it is D=D₁=D₂=0.002=0.2%.

[0028] To illustrate the meaning of the range between NE_(low) andNE_(high) refer to FIG. 1. FIG. 1 shows the likelihood density PD as afunction of the measured number of errors ne. In the example, the actualdetected number of errors ne within the measured sample of ns bits is10. The likelihood distribution of the errors is not known. The worstpossible likelihood distribution PD_(high) under all possible likelihooddistributions as well as the best possible likelihood distributionPD_(low) under all possible likelihood distributions are shown. Theworst possible likelihood distribution PD_(high) is characterized inthat the integral from 0 to ne=10 gives a total probability of D₁=0.002.The best possible likelihood distribution PD_(low) is characterized inthat the integral from ne=10 to ∞ gives a total probability of D₂=0.002.In the preferred embodiment D₁ is equal to D₂, i.e. D₁=D₂=0.002=0.2%.After having obtained the likelihood distribution PD_(high) and PD_(low)from formulas (1) and (2), the average values or mean values NE_(high)for the likelihood distribution PD_(high) and NE_(low) for thelikelihood distribution PD_(low) can be obtained. The range between themean value NE_(low) and NE_(high) is the confidence range CR indicatedin FIG. 1.

[0029] In the case the measured value ne is a rather untypical result(in the example just 0.2% probability) nevertheless the final result NEcan still be found in this range, called confidence range CR.

[0030] The probabilities D₁ and D₂ in (1) and (2) can be independent,but preferable they are dependent and equal (D=D₁=D₂)

[0031] For the Poisson distribution NE_(low) and NE_(high) can beobtained. With the formulas (3) and (4) respectively the inputs are thenumber of errors ne, measured in this test and the probabilities D andC=1−D. The Output is NE, the parameter describing the average of thePoisson distribution.

[0032] The following example is illustrated in FIG. 1 (D=D₁=D₂):$\begin{matrix}{{NE}_{low} = \frac{{qchisq}\left( {D,{2 \cdot {ne}}} \right)}{2}} & (3) \\{{NE}_{high} = \frac{{qchisq}\left( {C,{2\left( {{ne} + 1} \right)}} \right)}{2}} & (4)\end{matrix}$

[0033] Example:

[0034] Number of errors: ne=10

[0035] Probability: D=0.002 C=0.998

[0036] Result:

[0037] NE_(low)=3.26

[0038] NE_(high)=22.98

[0039] Interpretation:

[0040] Having measured ne=10 errors in a single test, then with a lowprobability D=0.002 the average number of errors NE in this test isoutside the range from 3.26 to 22.98 or with a high probability C=0.998inside this range from 3.26 to 22.98.

[0041] Such as the width of the Poisson distribution, the confidencerange CR increases proportional to SQR(ne), that means, it increasesabsolutely, but decreases relatively.

[0042] If the entire confidence range CR, calculated from a singleresult ne, is found on the good side (NE_(limit)>NE_(high)) of thespecified limit NE_(limit) we can state: With high probability C, thefinal result NE is better than the limit NE_(limit). Whereby NE_(limit)is given by

NE _(limit) =BER _(limit) ·ns   (5)

[0043] and BER_(limit) is the Bit Error Rate allowable for the deviceand obtained by an ideal long test with an infinite high number of bitsamples ns.

[0044] If the entire confidence range CR, calculated from a singleresult ne, is found on the bad side (NE_(limit)<NE_(low)) of thespecified limit NE_(limit) we can state: With high probability C, thefinal result NE is worse than the limit.

[0045] With each new sample and/or error a new test is considered,reusing all former results. With each new test the preliminary data forns, ne and ber is updated. For each new test the confidence range CR iscalculated and checked against the test limit NE_(limit).

[0046] Once the entire confidence range CR is found on the good side ofthe specified limit (NE_(limit)>NE_(high)), an early pass is allowed.Once the entire confidence range CR is found on the bad side of thespecified limit (NE_(limit)<NE_(low)) an early fail is allowed. If theconfidence range CR is found on both sides of the specified limit(NE_(low)<NE_(limit)<NE_(high)) it is evident neither to pass nor tofail the DUT early.

[0047]FIG. 1 illustrates the above conditions. Of course, NE_(limit) isa fixed value not altering during the test, but NE_(low) and NE_(high)as well as the confidence range CR are altering during the test. Forreasons of illustration, however, the three possibilities of thepossible positions of the confidence range CR with respect to theconstant limit NE_(limit) are drawn for the same example in FIG. 1.

[0048] The above can be described by the following formulas:

[0049] The current number of samples ns is calculated from thepreliminary Bit Error Ratio ber and the preliminary number of errors ne

ber=ne/ns   (6)

[0050] After a full test the final Bit Error Ratio is

BER_(limit)=NE_(limit)/ns   (7)

[0051] for abbreviation in the formula:

ber _(norm) =ber/BER _(limit) =ne/NE _(limit) (normalised ber)   (8)

[0052] Early pass stipulates:

NE_(high)<NE_(limit)   (9)

[0053] Early fail stipulates:

NE_(low)>NE_(limit)   (10)

[0054] Formula for the early pass limit: $\begin{matrix}{{ber}_{norm} = \frac{ne}{{NE}_{high}}} & (11)\end{matrix}$

[0055] This is the lower curve (bernorm_(pass) (ne, C)) in FIG. 2, whichshows ber_(norm) as a function of ne.

[0056] Formula for the early fail limit: $\begin{matrix}{{ber}_{norm} = \frac{ne}{{NE}_{low}}} & (12)\end{matrix}$

[0057] This is the upper curve (bernorm_(fail) (ne, D)) in FIG. 2.

[0058] As the early pass limit is not defined for ne=0 (normally thecase at the very beginning of the test for a good DUT), an artificialerror event with the first sample can be introduced. When the first realerror event occurs, the artificial error is replaced by this real one.This gives the shortest possible measurement time for an ideal good DUT.For example ns=5000 for BER_(limit)=0.001 and probability D=D₁=D₂=0.2%.

[0059] As the early fail limit uses NE_(low)<1 for small ne<k (in theexample below k=5) due to a decision problem at a fractional error, theearly fail limit at ne=k is extended with a vertical line upwards. Thisensures that a broken DUT hits the early fail limit in any case after afew samples, approx. 10 in the example. In other words, the test is notstopped as long as ne is smaller than k.

[0060] With each new sample and/or error a new test is considered,reusing all former results. With each new test the preliminary data forns, ne and ber and ber_(norm) are updated and a ber_(norm)/ne coordinateis entered into the ber_(norm)-diagram. This is shown in FIG. 3. Oncethe trajectory crosses the early fail limit (ber_(norm) (ne,D)) or theearly pass limit (ber_(norm) (ne,C)) the test may be stopped and theconclusion of early fail or early pass may be drawn based on thisinstant.

[0061]FIG. 3 shows the curves for early fail and early pass. ber_(norm)is shown as a function of the number of errors ne.

[0062] For the simple example demonstrated in FIG. 3, it isBER_(limit)=0.2=1/5 and the final Bit Error Ratio BER=0.25 (1/4). Thetest starts with the first bit sample, for which no error is detected.For the second sample, a first error is detected and the preliminary BitError Ratio ber=ne/ns=1/2 and ber_(norm)=ber/BER_(limit) becomes1/2:1/5=5/2. ber_(norm) after the second sample is marked with a cross ain FIG. 3. For the third, fourth and fifth sample, no further erroroccurs and ber_(norm) subsequently becomes 5/3, 5/4 and 5/5,respectively, which is marked with the crosses b, c and d in FIG. 3,respectively. The sixth sample brings a new error and ne becomes 2.Consequently, ber=ne/ns becomes 2/6 and ber_(norm) becomes 10/6. Thissituation is marked with cross e in FIG. 3. For the seventh; eighth andninth sample, no further error occurs and the situation after theseventh, eighth and ninth sample is marked with crosses f, g, h in FIG.3, respectively. The tenth sample brings a third error. Consequently,ber becomes 3/10 and ber_(norm) becomes 15/10. This situation is markedwith cross i in FIG. 3. As can be seen from FIG. 3, the trajectory isbetween the early fail curve and the early pass curve at the beginningof the test, but converges to a line Z, which crosses the early failcurve after about forty errors. After forty errors, it can thus bedecided that the tested DUT early fails the test.

[0063] If no early stop occurs the BER test may be stopped, after thefollowing condition is valid:

ne>=K   (13)

[0064] and the DUT shall be passed, if ns is sufficiently high. K is amaximum number of errors. For example K can be 200.

[0065] If the trajectory neither crosses the early fail curve nor theearly pass curve after K (for example 200) errors have occurred, the DUTcan be finally passed. If the DUT, however, is rather good or ratherbad, the tests can be stopped much earlier, long before the K=200 errorshave occurred. This significantly shortens the total test time.

[0066] In the above embodiment early fail means: a DUT is failed and aprobability of 0.2 % that it is actually better than the limit isaccepted. Further early pass means: the DUT is passed and a probabilityof 0.2 % that it is actually worse than the limit is accepted. If thetest is stopped at 200 errors the DUT is passed without any early failor early pass condition arbitrarily. It can cross the vertical 200 errorline in FIG. 2 at different heights, each height is connected with acertain statistical interpretation: The probability to have a DUT better(worse) than the limit is indicated in FIG. 5. The vertical in FIG. 5shows the position in FIG. 2 at the end of the test. The horizontal inFIG. 5 shows the respective probability.

[0067] This embodiment contains a pseudo paradox, due to statisticalnature of the test and limited test time, demonstrated with thefollowing example: A DUT, which is early failed, would pass if theprobability for a wrong decision has been reduced by widening the earlystop limits, or vice versa, if the test is stopped at 200 errors and theDUT is arbitrarily failed. A DUT, which is early passed, would fail ifthe probability for a wrong decision has been reduced.

[0068] The following embodiment resolves this pseudo paradox andadditionally accelerates the test. This is done by a meaningfulredefinition of the early pass limit maintaining the early fail limit.Early pass means now: A DUT is passed and a probability of 0.2 % that itis actually worse than M times the specified limit (M>1) is accepted.This is a worse DUT limit. This shifts the early pass limit upwards inFIG. 2 as shown in FIG. 4. Bernorm_(pass) (ne, C) in FIG. 2 becomesbernormbad_(pass) (ne, C) in FIG. 4. Bernorm_(fail) (ne, D) remainsunchanged. Now it is

NE _(limit,M) =BER _(limit) ·M·ns   (14)

[0069] and an early pass is allowed, ifNE_(limit)·M=NE_(limit,M)>NE_(high).

[0070] There are three high level parameters for the test:

[0071] Probability to make a wrong decision (proposal: C=0.2 %)

[0072] Final stop (proposal: K=200 errors)

[0073] Definition of a Bad DUT (BER_(limit)·M)

[0074] These parameters are interdependent. It is possible to enter twoof them and to calculate the third one. To make the test transparent, itis proposed to enter the wrong decision probability C and the final stopcondition K and to derive the Bad DUT factor M. This is done in thefollowing manner: The early pass limit is shifted upwards by a factor ofM, such that the early fail and the early pass limit intersect at 200errors in the example as shown in FIG. 4. FIG. 4 also shows the testlimit obtained from the crossing point from the early fail curve and theeasy pass curve, corresponding to M_(test).

[0075] There are infinite possibilities to resolve the above mentionedparadox.

[0076] In the example above the open end between the limits wasoriginally declared pass, and time was saved by multiplying the earlypass limit with M (M>1), shifting it upwards such that the early failand the early pass curve intersect at 200 errors (200: example fromabove). Such only a DUT, bad with high probability, is failed (customerrisk).

[0077] The complementary method is: The open end between the limits isdeclared fail, and time is saved by multiplying the early fail limitwith m(0<m<1), shifting it downwards, such that the early fail andthe-early pass curve intersect at 200 errors (200: example from above).Such only a DUT, good with high probability, is passed (manufacturerrisk).

[0078] The compromise method is: The open end between the limits ispartitioned in any ratio: the upper part is declared fail and the lowerpart is declared pass. Time is saved by multiplying the early fail limitwith m (0<m<1) and such shifting it downwards and by multiplying theearly pass limit with M (M>1) and such shifting it upwards. So the earlyfail and the early pass curve intersect at 200 errors (200: example fromabove).

[0079] With given D₁ and D₂ the early fail curve and the early passcurves in FIG. 3 and FIG. 2 or FIG. 4 can be calculated before the testis started. During the test only ber_(norm)=ne/NE_(limit) has to becalculated and to be compared with the early pass limit and the earlyfail limit as explained with respect to FIG. 3 and FIG. 4. Thus, nointensive calculation has to be done during the test.

1. Method for testing the Error Ratio BER of a device against a maximalallowable Error Ratio BER_(limit) with an early pass criterion, wherebythe early pass criterion is allowed to be wrong only by a smallprobability D₁, with the following steps measuring ns bits of the outputof the device, thereby detecting ne erroneous bits of these ns bits,assuming that the likelihood distribution giving the distribution of thenumber of erroneous bits ni in a fixed number of samples of bits isPD(NE,ni), wherein NE is the average number of erroneous bits, obtainingPD_(high) from D₁ = ∫₀^(n  e)PD_(high)(NE_(high), ni)ni

wherein PD_(high) is the worst possible likelihood distributioncontaining the measured ne erroneous bits with the probability D₁,obtaining the average number of erroneous bits NE_(high) for the worstpossible likelihood distribution PD_(high), comparing NE_(high) withNE_(limit)=BER_(limit)·ns, if NE_(limit) is higher than NE_(high)stopping the test and deciding that the device has early passed the testand if NE_(limit) is smaller than NE_(high) continuing the test wherebyincreasing ns.
 2. Method for testing the Error Ratio BER according toclaim 1, characterized in that the likelihood distribution PD(NE,ni) isthe Poisson distribution.
 3. Method for testing the Error Ratio BERaccording to claim 1 or 2, characterized in that for avoiding aundefined situation for ne=0 starting the test with an artificial errorne=1 not incrementing ne then a first error occurs.
 4. Method fortesting the Error Ratio BER of a device against a maximal allowableError Ratio BER_(limit) with an early fail criterion, whereby the earlyfail criterion is allowed to be wrong only by a small probability D₂,with the following steps measuring ns bits of the output of the device,thereby *detecting ne erroneous bits of these ns bits, assuming that thelikelihood distribution giving the distribution of the number oferroneous bits ni in a fixed number of samples of bits is PD(NE,ni),wherein NE is the average number of erroneous bits, obtaining PD_(low)from the D₂ = ∫_(ne)^(∞)PD_(low)(NE_(low), ni)ni

wherein PD_(low) is the best possible likelihood distribution containingthe measured ne erroneous bits with, the probability D₂, obtaining theaverage number of erroneous bits NE_(low) for the best possiblelikelihood distribution PD_(low), comparing NE_(low) withNE_(limit)=BER_(limit)·ns, if NE_(limit) is smaller than NE_(low)stopping the test and deciding that the device has early failed the testand if NE_(limit) is higher than NE_(low) continuing the test wherebyincreasing ns.
 5. Method for testing the Error Ratio BER according toclaim 4, characterized in that the likelihood distribution PD(NE,ni) isthe Poisson distribution.
 6. Method for testing the Error Ratio BERaccording to claim 4 or 5, characterized in that for avoiding aundefined situation for neck, wherein k is a small number of errors, notstopping the test as long as ne is smaller than k.
 7. Method for testingthe Error Ratio BER according to claim 6, characterized in that k is 5.8. Method for testing the Error Ratio BER according to any of claims 4to 7, characterized by an additional early pass criterion, whereby theearly pass criterion is allowed to be wrong only by a small probabilityD₁ with the following additional steps assuming that the likelihooddistribution giving the distribution of the number of erroneous bits niin a fixed number of samples of bits is PD(NE,ni), wherein NE is theaverage number of erroneous bits, obtaining PD_(high) fromD₁ = ∫₀^(n  e)PD_(high)(NE_(high), ni)ni

wherein PD_(high) is the worst possible likelihood distributioncontaining the measured ne erroneous bits with the probability D₁,obtaining the, average number of erroneous bits NE_(high) for the worstpossible likelihood distribution PD_(high), comparing NE_(high) withNE_(limit)=BER_(limit)·ns, if NE_(limit) is higher than NE_(high)stopping the test and deciding that the device has early passed the testand if NE_(limit) is smaller than NE_(high) continuing the test, wherebyincreasing ns.
 9. Method for testing the Error Ratio BER according toclaim 8, characterized in that for avoiding a undefined situation forne=0 starting the test with an artificial error ne=1 not incrementing nethen a first error occurs.
 10. Method for testing the Error Ratio BERaccording to claim 8 or 9, characterized in that the probability D₁ forthe wrong early pass criterion and the probability D₂ for the wrongearly fail criterion are equal (D₁=D₂).
 11. Method for testing the ErrorRatio BER according to any of claims 4 to 7, characterized by anadditional early pass criterion, whereby the early pass criterion isallowed to be wrong only by a small probability D₁ with the followingadditional steps assuming that the likelihood distribution giving thedistribution of the number of erroneous bits ni in a fixed number ofsamples of bits is PD(NE,ni), wherein NE is the average number oferroneous bits, obtaining PD_(high) fromD₁ = ∫₀^(n  e)PD_(high)(NE_(high), ni)ni

wherein PD_(high) is the worst possible likelihood distributioncontaining the measured ne erroneous bits with the probability D₁,obtaining the average number of erroneous bits NE_(high) for the worstpossible likelihood distribution PD_(high), comparing NE_(high) withNE_(limit,M)=BER_(limit)·M·ns, with M>1 if NE_(limit,M) is higher thanNE_(high) stopping the test and deciding that the device has earlypassed the test and if NE_(limit,M) is smaller than NE_(high) continuingthe test, whereby increasing ns.